Display device and display method that determines intention or status of a user

ABSTRACT

The present invention provides a display apparatus and a display method for realizing control for display operations by a user precisely reflecting the user&#39;s status, i.e., the user&#39;s intentions, visual state and physical conditions. Worn as an eyeglass-like or head-mount wearable unit for example, the display apparatus of the present invention enables the user to recognize visibly various images on the display unit positioned in front of the user&#39;s eyes thereby providing the picked up images, reproduced images, and received images. As control for various display operations such as switching between the display state and the see-through state, display operation mode and selecting sources, the display apparatus of the present invention acquires information about either behavior or physical status of the user, and determines either intention or status of the user in accordance with the acquired information, thereby controlling the display operation appropriately on the basis of the determination result.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/481,576, filed Sep. 9, 2014 which is a continuation of applicationSer. No. 13/614,591, filed Sep. 13, 2012 (now U.S. Pat. No. 8,860,867),which is a continuation of U.S. patent application Ser. No. 12/438,735,filed Feb. 25, 2009 (now U.S. Pat. No. 8,368,794), which is a nationalstage of International patent application No. PCT/JP2007/65083, filedAug. 1, 2007, which claims priority to Japanese patent application No.2006-244686, filed Sep. 8, 2006. The entire contents of each of whichare incorporated herein by reference.

DISPLAY APPARATUS AND DISPLAY METHOD

1. Technical Field

The present invention relates to a display apparatus which is worn by auser as part of an eyeglass-like or head-mount wearable unit and whichhas display means positioned in front of the user's eyes for imagedisplay purposes, as well as to a display apparatus for use with thatdisplay apparatus.

2. Background Art

A variety of wearable display apparatuses have so far been proposed, asdescribed illustratively in Japanese Patent Laid-open Nos. Hei 8-126031,Hei 9-27970 and Hei 9-185009. Patent Documents disclose displayapparatuses each worn by a user as part of an eyeglass-like orhead-mount wearable unit with a display unit located immediately beforethe user's eyes for image display.

However, traditional display apparatuses such as those cited above haveyet to relieve the user of the need to operate their keys or othercontrols. These apparatuses have yet to offer the kind of display thatwould suitably reflect the user's intentions and biometric status.

The present invention has been made in view of the above circumstancesand provides an apparatus and a method whereby display operations areperformed in a manner precisely reflecting the user's status, i.e., hisor her intentions and physical conditions.

DISCLOSURE OF THE INVENTION

In carrying out the present invention and according to one embodimentthereof, there is provided a display apparatus including: display meansfor displaying images as it is positioned in front of the eyes of auser; user information acquisition means for acquiring information abouteither behavior or physical status of the user; and control means fordetermining either intention or status of the user in accordance withthe information acquired by the user information acquisition means, thecontrol means further controlling display operations of the displaymeans in a manner reflecting the result of the determination.

Preferably, the display apparatus according to an embodiment of thepresent invention may further include image pickup means for picking upimages. In this structure, the control means may preferably control thedisplay means to display image data picked up by the image pickup meansin accordance with the information acquired by the user informationacquisition means.

Preferably, the display apparatus of the present invention may furtherinclude reproduction means for reproducing data from a recording medium.In this structure, the control means may preferably control the displaymeans to display the data reproduced by the reproduction means inaccordance with the information acquired by the user informationacquisition means. The control means may preferably control reproductionoperations of the reproduction means in accordance with the informationacquired by the user information acquisition means while the datareproduced by the reproduction means is being displayed by the displayunit.

Preferably, the display apparatus according to an embodiment of thepresent invention may further include reception means for receiving datathrough communication with an external apparatus. In this structure, thecontrol means may preferably control the display means to display thedata received by the reception means in accordance with the informationacquired by the user information acquisition means.

Preferably, the display means may be capable of switching between asee-through state and a display state, the see-through state being astate in which the display means remains either transparent ortranslucent, the display state being a state in which the display meansdisplays supplied data.

The user information acquisition means may preferably be a sensorconfigured to detect acceleration, angular velocity, or vibration.

The user information acquisition means may preferably be a sensorconfigured to detect movements of the head, arms, hands, legs, or wholebody of the user.

The user information acquisition means may preferably be a sensorconfigured to detect a stationary state, a walking state, and a runningstate of the user.

The user information acquisition means may preferably be a visual sensorconfigured to detect visual information about the user.

The user information acquisition means may preferably be a sensorconfigured to detect, as visual information about the user, the line ofsight, focal distance, pupil dilation, retinal pattern, or blinks of theuser.

The user information acquisition means may preferably be a biometricsensor configured to detect biometric information about the user.

The user information acquisition means may preferably be a sensorconfigured to detect, as biometric information about the user, the heartrate, pulse rate, perspiration, brain wave, galvanic skin reflex, bloodpressure, body temperature, or breathing rate of the user.

The user information acquisition means may preferably be a biometricsensor configured to detect information about a tense state or anexcited state of the user.

The control means may preferably control the display means to start andend the display operations thereof.

The control means may preferably control the display means to switchbetween the see-through state and the display state mentioned above.

The control means may preferably control switching between sources whichsupply data to the display means for display.

The control means may preferably control the display means to scale upand down a display image.

The control means may preferably control the display means to split anonscreen display.

The control means may preferably control the display means to controldisplay brightness.

The control means may preferably control signal processing of an imagesignal to be displayed by the display means.

The control means may preferably control the display means to scrollimages or to feed pages.

The control means may preferably determine an input operation regardingthe image displayed by the display means in accordance with theinformation acquired by the user information acquisition means.

According to another embodiment of the present invention, there isprovided a display method for use with a display apparatus havingdisplay means for displaying images as it is positioned in front of theeyes of a user, and user information acquisition means for acquiringinformation about either behavior or physical status of the user, thedisplay method including the steps of: acquiring information about thebehavior or physical status of the user; and determining eitherintention or status of the user in accordance with the informationacquired in the information acquiring step, the determining step furthercontrolling display operations of the display means in a mannerreflecting the result of the determination.

Where the present invention embodied as outlined above is in use, theinventive display apparatus is worn by the user as part of aneyeglass-like or head-mount wearable unit having display meanspositioned in front of the user's eyes for visual recognition. Wearingthe display apparatus allows the user to view what is being displayed bythe display means such as images picked up by image pickup means, imagesconstituted by the data reproduced by reproduction means, or imagescomposed of the data received by reception means of the apparatus.

It is preferred that the display operations be controlled in a mannersuitably reflecting the user's intention and status. More specifically,the display operations may involve causing the display means to switchbetween the display state and the see-through state; selecting sourcesfrom which to acquire the image data to be displayed; adjusting diversedisplay settings such as splitting of the onscreen display, scale-up andscale-down of display images, control of display brightness and otherdisplay quality-related attributes, scrolling of images and feeding ofpages; inputting data to the display means for display; and causing thedisplay means to display reproduced images. The present inventionproposes that these operations be carried out not by the usermanipulating switches and controls but by the display apparatus itselfacquiring information about the user's behavior or physical status,determining the user's intention or status based on the acquiredinformation, and executing suitable controls in accordance with theresult of the determination.

According to an embodiment of the present invention, as outlined above,the display apparatus determines the user's intention or status based onthe information about the user's behavior or physical status andcontrols the display operations accordingly while the display means isdisplaying images immediately before the user's eyes. This structurepermits appropriate display performance reflecting the user's intentionor status without bothering the user to manipulate controls or switches.The inventive display apparatus thus provides excellent ease of use forthe user while offering varieties of visual experiences.

Since the display means can be placed in the transparent or translucentstate as desired, the user can lead normal daily life while continuouslywearing the display apparatus. The advantages of the inventive apparatuscan thus be exploited by the user while maintaining his or her usuallifestyle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a external view explanatory of a display apparatus embodyingthe present invention.

FIGS. 2a-2d are schematic views explanatory of how the inventive displayapparatus is typically configured.

FIG. 3 is a block diagram showing a typical structure of the displayapparatus.

FIG. 4 is a block diagram showing another typical structure of thedisplay apparatus.

FIGS. 5a-5c are schematic views explanatory of a see-through state andan image display state in which the display apparatus may be placed.

FIGS. 6a-6c are schematic views explanatory of typical display imagesretrieved from a storage unit of the display apparatus.

FIGS. 7a-7c are schematic views explanatory of typical display imagescoming from a communication unit of the display apparatus.

FIGS. 8a-8c are schematic views explanatory of other display imagescoming from the communication unit of the display apparatus.

FIGS. 9a-9c are schematic views explanatory of how onscreen displays aresplit by the display apparatus.

FIGS. 10a-10b are schematic views explanatory of how an image is scaledup when displayed.

FIGS. 11a-11b are schematic views explanatory of how the brightness of adisplay image is adjusted.

FIG. 12 is a flowchart of steps constituting a typical control processperformed by the display apparatus.

FIGS. 13a-13c are flowcharts of steps constituting typical processesperformed by the display apparatus to recognize a display start trigger.

FIG. 14 is a flowchart of steps constituting a typical process performedby the display apparatus to recognize display control triggers.

FIG. 15 is a flowchart of steps constituting a typical process performedby the display apparatus to recognize another display control trigger.

FIG. 16 is a flowchart of steps constituting a typical process performedby the display apparatus to recognize other display control triggers.

FIG. 17 is a flowchart of steps constituting a typical process performedby the display apparatus to recognize other display control triggers.

FIGS. 18a-18b are flowcharts of steps constituting typical processesperformed by the display apparatus to recognize a display end trigger.

FIGS. 19a-19b are flowcharts of steps constituting other typicalprocesses performed by the display apparatus to recognize the displayend trigger.

BEST MODE FOR CARRYING OUT THE INVENTION

The display apparatus and display method practiced as preferredembodiments of the present invention will now be described under thefollowing headings:

-   -   [1. Appearance of the display apparatus and its relations to        external apparatuses]    -   [2. Typical structure the display apparatus]    -   [3. Typical displays]    -   [4. Determination of user status]    -   [5. Various operations]    -   [6. Effects, variations and extensions of the embodiments]

1. Appearance of the Display Apparatus and its Relations to ExternalApparatuses

FIG. 1 shows an external view of a display apparatus 1 embodying thepresent invention as an eyeglass-like display. The display apparatus 1typically has a wearable semicircular frame structure which covers thetemples and back of a user's head and which is worn by the user in amanner hitched onto his or her ear conches as illustrated.

Worn by the user as shown in FIG. 1, the display apparatus 1 has a pairof display units 2 positioned immediately before the user's eyes, i.e.,where the lenses of a pair of eyeglasses would ordinarily be located.The display unit 2 illustratively utilizes a liquid crystal display(LCD) panel of which the permeability is adjusted so as to bring about asee-through state (i.e., transparent or translucent state) as needed.Placed in its see-through state, each display unit 2 poses no impedimentto the user's daily life when worn continuously just like eyeglasses.

With the display unit 2 worn by the user, an image pickup lens 3 a isoriented forward so as to pick up images of objects in the user's lineof sight.

A lighting element 4 a is positioned in a manner giving illumination inthe direction in which the image pickup lens 3 a picks up the images.The lighting element 4 a is typically formed by a light-emitting diode(LED).

A pair of earphone-like speakers 5 a are furnished to be inserted intothe user's ears (only the left-hand side is shown in FIG. 1).

A pair of microphones 6 a and 6 b for picking up external sounds arepositioned at a right-hand edge of the display unit 2 for the right eyeand at a left-hand edge of the display unit 2 for the left eye.

What is shown in FIG. 1 is only an example; there may be diversestructures in which the display apparatus 1 may be devised to be worn bythe user. Generally, the wearable unit may be of eyeglass type or ofhead mount type. Any structure is acceptable for the embodiment of theinvention as long as the display unit 2 is located in front of and closeto the user's eyes. The pair of display units 2 above may be replacedalternatively by a single display unit 2 for one-eye use.

As another alternative, the pair of earphone-like speakers 5 a forstereo sound effects may be replaced by a single earphone for one-earuse. Likewise, the pair of microphones 6 a and 6 b may be replaced by asingle microphone.

The structure of FIG. 1 was described above as inclusive of thecapability to pick up images.

Alternatively, the structure may dispense with the image pickupfunction. As another alternative, the display apparatus 1 may be free ofmicrophones and/or earphone-like speakers. As a further alternative, thedisplay apparatus 1 may dispense with the lighting element 4 a.

The internal structure of the display apparatus 1, which will bediscussed later in detail, may include a reproduction capability(storage unit 25 in FIGS. 3 and 4) for reproducing data from a recordingmedium and a communication capability (communication unit in FIGS. 3 and4) for communicating with an external apparatus.

Where these capabilities are implemented, the sources from which toderive data as the images to be displayed by the display unit 2 mayinclude an image pickup capability block, a reproduction capabilityblock, and a communication capability block.

FIG. 2 illustrates how the display apparatus 1 is configured in relationto externally connected apparatuses.

(a) of FIG. 2 shows a case in which the display apparatus 1 isconfigured for standalone use. In this case, if the display apparatus 1has the image pickup capability, the image data picked up thereby may bedisplayed by the display unit 2. If the display apparatus 1 has thereproduction capability, then the image data reproduced from therecording medium may be reproduced by the display unit 2. The data thatmay be reproduced from the recording medium for display purposes may beof diverse kinds: moving image contents such as movies and video clips;still image contents picked up by digital still cameras or the like andstored on the recording medium; electronic book data; computer use dataprepared by the user on his or her PC and recorded to the recordingmedium, including image data, text data, and spreadsheet data; videogame images derived from video game programs recorded on the recordingmedium; and any other data that may be recorded to the recording mediumand reproduced therefrom for display use.

(b) of FIG. 2 shows a case where the display apparatus 1 is configuredto have the communication capability communicating with an externalimage pickup apparatus 70. In this case, the display apparatus 1receives images (moving/still images) picked up by the image pickupapparatus 70 and causes the display unit 2 to display the receivedimages. The external image pickup apparatus 70 may typically be a videocamera or a digital still camera capable of data communication.Alternatively, another display apparatus 1 having the image pickupcapability as shown in FIG. 1 may be set up as an external image pickupapparatus arranged to communicate with this display apparatus 1.

The external image pickup apparatus 70 may be of diverse types. It maybe an image pickup apparatus in the possession of the user utilizing thedisplay apparatus 1 or an image pickup apparatus owned by anacquaintance of the user of the display apparatus 1. Alternatively, theexternal image pickup apparatus 70 may be an apparatus set up as part ofthe facility of an image-providing public service or a piece ofequipment established by a private service-offering company, bothcapable of communicating with the display apparatus 1.

(c) of FIG. 2 shows a case where the display apparatus 1 has thecommunication capability communicating with an external content sourceapparatus 71. In this case, the display apparatus 1 receives images(moving/still images) from the content source apparatus 71 and causesthe display unit 2 to display the received images.

Typically, the content source apparatus 71 may be an AV (audio visual)apparatus such as a video unit, a TV tuner or a home server; or aninformation processing apparatus such as a PC (personal computer), PDA(personal digital assistant), or a mobile phone. As in the case above,the content source apparatus 71 may be an apparatus in the possession ofthe user handling the display apparatus 1, an apparatus owned by anacquaintance of the user, an apparatus set up as part of the facility ofa content-providing public service, or a server established by a privatecontent-offering company.

The data sent from the content source apparatus 71 to the displayapparatus 1 for display purposes may also be of diverse kinds: movingimage contents such as movies and video clips; still image contentspicked up by digital still cameras or the like and stored on therecording medium; electronic book data; computer use data prepared bythe user on his or her PC and recorded to the recording medium,including image data, text data, and spreadsheet data; video game imagesderived from video game programs recorded on the recording medium; andany other data that may be available in suitable format for display use.

(d) of FIG. 2 shows a case in which the display apparatus 1 has thecommunication capability, especially one with a communication accessfeature which makes use of a network 73 such as the Internet, thecapability allowing the display apparatus 1 to communicate with anexternally established image pickup apparatus 70 or content sourceapparatus 71 over the network 73. In this case, the display apparatus 1receives diverse kinds of data through the network 73 and causes thedisplay unit 2 to display the images of the received data.

2. Typical Structure the Display Apparatus

FIG. 3 shows a typical internal structure of the display apparatus 1.

A system controller 10 is typically composed of a microcomputerincluding a CPU (central processing unit), a ROM (read only memory), aRAM (random access memory), a nonvolatile memory, and an interface. Inoperation, the system controller 10 controls the display apparatus 1 asa whole.

The system controller 10 controls the components of the displayapparatus 1 in accordance with the user's status. That is, the systemcontroller 10 runs on a program that detects and determines the user'sstatus and controls the internal components in their operation in amanner reflecting what is detected and determined. In functional terms,the system controller 10 thus has two capabilities: a user statusdetermination capability 10 a to determine the user's status, and anoperation control capability 10 b to give control instructions to thecomponents in accordance with the result of the determination by theuser status determination capability 10 a, as shown in FIG. 3.

The display apparatus 1 has an image pickup unit 3, an image pickupcontrol unit 11, and an image pickup signal processing unit 15 forpicking up images of the scene in front of the user.

The image pickup unit 3 has a lens block, a lens drive block, and asolid state imaging device array. The lens block is made up of the imagepickup lens 3 a (FIG. 1), an aperture, a zoom lens, and a focus lens.The lens drive block is designed to drive the lenses in focus and zoomoperations. The solid sate imaging device array detects the light pickedup by the lens block and photoelectrically converts the detected lightinto an image pickup signal. Typically, the solid sate imaging devicearray is a CCD (charge coupled device) sensor array or a CMOS(complementary metal oxide semiconductor) sensor array.

The image pickup signal processing unit 15 includes a video A/Dconverter, and a sample-and-hold/AGC (automatic gain control) circuitthat performs gain control and waveform shaping on the signal acquiredby the solid state imaging devices of the image pickup unit 3. Workingin this manner, the image pickup signal processing unit 15 obtains animage pickup signal in digital data form. The image pickup signalprocessing unit 15 also carries out white balance control, brightnesscontrol, white signal adjustment, and camera shake correction on theimage pickup signal.

Under control of the system controller 10, the image pickup control unit11 controls the image pickup unit 3 and image pickup signal processingunit 15 in operation. Illustratively, the image pickup control unit 11controls activation and deactivation of the image pickup unit 3 andimage pickup signal processing unit 15. The image pickup control unit 11also effects motor control on the image pickup unit 3 to regulate autofocusing, automatic exposure, aperture opening, and zoom functions.

The image pickup control unit 11 has a timing generator. Timing signalsgenerated by the timing generator are used to control the solid stateimaging devices, as well as the sample-and-hold/AGC circuit and thevideo A/D converter in the image pickup signal processing unit 11 intheir signal processing. Image pickup frame rate can also be controlledas part of the timing control scheme.

Furthermore, the image pickup control unit 11 controls the solid stateimaging devices and the image pickup signal processing unit 15 in termsof imaging sensitivity and signal processing. Illustratively, as part ofthe imaging sensitivity control scheme, gain control is effected on thesignal read from the solid state imaging devices. Controls are likewiseperformed on the black level setting, on coefficients used in imagepickup signal processing in the digital data stage, and on the amount ofcorrection in camera shake correction processing. In terms of imagingsensitivity, it is possible to carry out overall sensitivity adjustmentsregardless of wavebands and to perform sensitivity adjustments takingspecific wavebands into consideration such as infrared and ultravioletranges. Sensitivity adjustments dependent on wavelengths can be broughtabout by the introduction of a wavelength filter in the image pickuplens block and by the execution of wavelength filter calculations on theimage pickup signal. More specifically, the image pickup control unit 11can effect sensitivity control by suitably adjusting the wavelengthfilter being inserted and by designating appropriate filter calculationcoefficients.

The image pickup signal (acquired image data) picked up by the imagepickup unit 3 and processed by the image pickup signal processing unit15 is supplied to an image input/output control unit 27.

Under control of the system controller 10, the image input/outputcontrol unit 27 controls the transfer of image data, specificallybetween the image pickup block (image pickup signal processing unit 15),the display block (display image processing unit 12), a storage unit 25,and a communication unit 26.

Illustratively, the image input/output control unit 27 supplies thedisplay image processing unit 12, storage unit 25, and communicationunit 26 with the image data constituting the image pickup signalprocessed by the image pickup signal processing unit 15.

The image input/output control unit 27 also supplies the display imageprocessing unit 12 and communication unit 26 with the image datareproduced illustratively from the storage unit 25.

Furthermore, the image input/output control unit 27 supplies the displayimage processing unit 12 and storage unit 25 with the image datareceived illustratively by the communication unit 26.

The display apparatus 1 has the display unit 2, display image processingunit 12, a display drive unit 13, and a display control unit 14 makingup the structure for offering displays to the user.

Illustratively, the image data picked up by the image pickup unit 3 andprocessed by the image signal processing unit 15 as the image pickupsignal may be sent to the display image processing unit 12 through theimage input/output control unit 27. The display image processing unit 12is what is generally referred to as a video processor that carries outvarious display-related processes on the supplied image data. Theseprocesses include brightness level adjustment, color correction,contrast adjustment, and sharpness control (edge enhancement) effectedon the image data. The display image processing unit 12 is furthercapable of generation of a partially scaled-up or scaled-down image fromthe supplied image data, splitting and superimposing of display imagesbased on the supplied image data, generation of characters and graphicimages, and superimposing of a generated image onto the supplied imagedata. That is, the supplied image data can be processed in diversefashion by the display image processing unit 12.

The display drive unit 13 is constituted by pixel drive circuits thatenable the display unit 2 (e.g., LCD) to display the image data suppliedfrom the display image processing unit 12. More specifically, the drivecircuits furnish the pixels arranged in matrix fashion in the displayunit 2 with drive signals based on video signals occurring atpredetermined horizontal and vertical drive timings, whereby the imageof the supplied data is displayed. The display drive unit 13 may placethe display unit 2 in the see-through state by suitably controlling thetransmittance of each of the pixels in the unit 2.

The display control unit 14 controls the display image processing unit12 in processing and the display drive unit 13 in operation undercontrol of the system controller 10. More specifically, the displaycontrol unit 14 causes the display image processing unit 12 to carry outthe above-described processes and enables the display drive unit 13 toswitch between the see-through state and the image display state.

The image data reproduced from the storage unit 25 or the image datareceived by the communication unit 26 may be supplied to the displayimage processing unit 12 via the image input/output control unit 27. Insuch cases, the reproduced image or the received image is output by thedisplay unit 2 thanks to the above-described workings of the displayimage processing unit 12 and display drive unit 13.

The display apparatus 1 also includes an audio input unit 6, an audiosignal processing unit 16, and an audio output unit 5.

The audio input unit 6 is made up of the microphones 6 a and 6 b shownin FIG. 1, and of microphone amplifiers and A/D converters foramplifying and converting the audio signals obtained by the microphones6 a and 6 b. An audio signal derived from such processing is output asaudio data.

The audio data acquired by the audio input unit 6 is supplied to theaudio input/output control unit 28.

Under control of the system controller 10, the audio input/outputcontrol unit 28 controls transfer of the supplied audio data,specifically the transfer of audio signals between the audio input unit6, audio signal processing unit 16, storage device 25, and communicationunit 26.

For example, the audio input/output control unit 28 supplies the audiodata obtained by the audio input unit 6 to the audio signal processingunit 16, to the storage unit 25, or to the communication unit 26.

The audio input/output control unit 28 further supplies the audio datareproduced illustratively from the storage unit 25 to the audio signalprocessing unit 16 or to the communication unit 26.

The audio input/output control unit 28 also forwards the audio datareceived illustratively by the communication unit 26 to the audio signalprocessing unit 16 or to the storage unit 25.

The audio signal processing unit 16 is typically constituted by adigital signal processor and a D/A converter. The audio data obtained bythe audio input unit 6 or the audio data coming from the storage unit 25or from the communication unit 26 is supplied to the audio signalprocessing unit 16 through the audio input/output control unit 28. Undercontrol of the system controller 10, the audio signal processing unit 16performs such processes as sound level control, sound quality control,and acoustic effect adjustments on the supplied audio data. The audiodata thus processed is converted to an analog signal that is sent to theaudio output unit 5. The audio signal processing unit 16 is not limitedto the structure for digital signal processing; the unit 16 may also beimplemented in a structure including analog amplifiers and analogfilters for analog signal processing.

The audio output unit 5 is made up of the above-described pair ofearphone-like speakers 5 a in FIG. 1 and of amplifier circuitscorresponding to the earphone-like speakers 5 a.

The audio input unit 6, audio signal processing unit 16, and audiooutput unit 5 combine to let the user listen to external sounds, to thesounds reproduced from the storage unit 25, or to the sounds received bythe communication unit 26. The audio output unit 5 may be structuredalternatively as so-called bone conduction speakers.

The storage unit 25 allows data to be written to and read from asuitable recording medium. Typically, the storage unit 25 is implementedas a hard disk drive (HDD). Obviously, the recording medium may be ofdiverse types including a solid state memory such as a flash memory, amemory card containing a solid state memory, an optical disk, amagneto-optical disk, and a hologram memory. Any of such recording mediamay be adopted as long as the storage unit 25 can properly write andread data to and from the selected medium.

The image data picked up by the image pickup unit 3 and processed by theimage pickup signal processing unit 15 as the image pickup signal, orthe image data received by the communication unit 26 can be sent to thestorage unit 25 via the image input/output control unit 27. The audiodata obtained by the audio input unit 6 or the image data received bythe communication unit 26 may be forwarded to the storage unit 25through the audio input/output control unit 28.

Under control of the system controller 10, the storage unit 25 encodesthe supplied image data or audio data preparatory to recording to therecording medium and then writes the encoded data to the recordingmedium.

The storage unit 25 also reproduces image data or audio data from therecording medium under control of the system controller 10. Thereproduced image data is output to the image input/output control unit27; the reproduced audio data is sent to the audio input/output controlunit 28.

The communication unit 26 sends and receives data to and from externalapparatuses. Diverse kinds of external apparatuses may communicate withthe communication unit 26, including the image pickup apparatus 70 andcontent source apparatus 71 shown in FIG. 2.

The communication unit 26 may communicate with the external apparatusover a network such as a wireless LAN or a Bluetooth setup withshort-distance wireless links to network access points. Alternatively,the communication unit 26 may engage in direct wireless communicationwith an external apparatus having a suitable communication capability.

The image data picked up by the image pickup unit 3 and processed by theimage pickup signal processing unit 15 as the image pickup signal, orthe image data reproduced from the storage unit 25 can be sent to thecommunication unit 26 through the image input/output control unit 27.The audio data acquired by the audio input unit 6 or reproduced from thestorage unit 25 can be supplied to the communication unit 26 via theaudio input/output control unit 28.

Under control of the system controller 10, the communication unit 26encodes and modulates the supplied image data or audio data preparatoryto transmission, and transmits the data thus processed to the externalapparatus.

The communication unit 26 further receives data from the externalapparatus. The image data received and demodulated by the communicationunit 26 is output to the image input/output control unit 27; the audiodata received and demodulated likewise is forwarded to the audioinput/output control unit 28.

The display apparatus 1 also includes a lighting unit 4 and a lightingcontrol unit 18. The lighting unit 4 is made up of the lighting element4 a shown in FIG. 1 and a lighting circuit for causing the lightingelement 4 a (e.g., LED) to emit light. The lighting control unit 18causes the lighting unit 4 to perform light-emitting operation undercontrol of the system controller 10.

With the lighting element 4 a of the lighting unit 4 positioned in amanner giving illumination in the forward direction, the lighting unit 4provides illumination in the user's line of sight.

The display apparatus 1 further includes a visual sensor 19, anacceleration sensor 20, a gyro 21, and biometric sensors 22 constitutingthe structure for acquiring user information.

The visual sensor 19 detects information about the user's vision.Illustratively, the visual sensor 19 is capable of detecting suchsight-related information as the line of sight, focal distance, pupildilation, retinal pattern, and blinks of the user.

The acceleration sensor 20 and gyro 21 output signals reflecting theuser's movements. Illustratively, these sensors detect the movements ofthe user's head, neck, whole body, arms, and legs.

The biometric sensors 22 detect biometric information about the user.Illustratively, the biometric sensors 22 detect the heart rate, pulserate, perspiration, brain wave, galvanic skin reflex (GSR), bloodpressure, body temperature, and breathing rate of the user. The signalsdetected by the biometric sensors 22 constitute information by which todetermine the user's diverse states: a tense state, an excited state, acalm state, a sleepy state, or a comfortable state, or an uncomfortablestate.

The visual sensor 19, acceleration sensor 20, gyro 21, and biometricsensors 22 combine to acquire information about the user wearing thedisplay apparatus 1 in connection with the user's behavior or physicalstatus. The user information thus acquired is supplied to the systemcontroller 10.

By operating its user status determination capability 10 a, the systemcontroller 10 determines the user's intention or status reflecting theacquired user information. With the user's intention or status thusdetermined, the system controller 10 controls display-related operationsthrough the processing of its operation control capability 10 b. Morespecifically, the system controller 10 instructs the display controlunit 14 to control the display image processing unit 12 and displaydrive unit 13 in operation, selects the source from which to obtain thedata to be displayed, or controls the storage unit 25 in reproduction orthe communication unit 26 in communication.

Although the visual sensor 19, acceleration sensor 20, gyro 21 andbiometric sensors 22 were shown to constitute the structure foracquiring user information, the installation of all these components isnot requisite. Other sensors may be added as well, including a voicesensor for detecting the user's voice and a lip-reading sensor forreading the user's lip movements.

FIG. 4 shows a typical structure of the display apparatus 1 minus theimage pickup capability. In FIG. 4 as well as in FIG. 3, like referencenumerals designate like or corresponding functional block elements, andtheir descriptions will be omitted where redundant from the ensuingdescription.

The structure in FIG. 4 is basically the same as that in FIG. 3 minusthe image pickup unit 3, image pickup signal processing unit 15, imagepickup control unit 11, lighting unit 4, lighting control unit 18, andaudio input unit 6.

In the structure of FIG. 3, there are three sources from which to tapthe data for display on the display unit 2: the image pickup capabilityblock (image pickup unit 3, image pickup signal processing unit 15,image pickup control unit 11), reproduction capability block (storageunit 25), and reception capability block (communication unit 26). Bycontrast, the structure of FIG. 4 has two sources that may supply datafor display on the display unit 2: the reproduction capability block(storage unit 25), and reception capability block (communication unit26).

In other words, the setup of FIG. 3 has three display image sources inthe display apparatus 1, while the setup of FIG. 4 contains two displayimage sources inside the display apparatus 1.

Although not shown, other structures than those in FIGS. 3 and 4 may bedevised, each containing a different number or different types ofdisplay image sources in the display apparatus 1. For example, analternative structure may be furnished with only the image pickupcapability block; another structure with only the reproductioncapability block; a further structure with the reception capabilityblock; an even further structure with both the image pickup capabilityblock and the reproduction capability block; and a still furtherstructure with the image pickup capability block and receptioncapability block.

3. Typical Displays

The system controller 10 controls display-related operations in responseto the user's intention or status, selecting the source that suppliesthe data to be displayed and carrying out processes on display images.This allows the user to view diverse forms and contents of displays onthe display unit 2. FIGS. 5 through 10 show typical displays.

(a) of FIG. 5 shows a case in which the display unit 2 is placed in thesee-through state. In this case, the display unit 2 simply acts as atransparent plate member through which the user views the actual scenein front.

(b) of FIG. 5 shows a case where the display unit 2 displays the imagepicked up by the image pickup unit 3. In this case, illustratively withthe user in the state of (a) of FIG. 5 above, the image pickup unit 3,image pickup signal processing unit 15, display image processing unit12, and display drive unit 13 work to get the display unit 2 displayinga picked-up image. The image displayed by the display unit 2 (normallypicked-up image) this time is approximately the same as that in thesee-through state. That is, the user is viewing the normal field ofvision provided by the image being picked up.

(c) of FIG. 5 shows a case in which the system controller 10 controlsthe pickup image control unit 11 to get the image pickup unit 3 pickingup a telephoto image. The telephoto image thus obtained is displayed bythe display unit 2.

Although not shown, if the system controller 10 causes the image pickupcontrol unit 11 to get the image pickup unit 3 picking up a wide-angleshot image, then that wide-angle shot will be displayed by the displayunit 2. Switchover between telephoto and wide-angle shot settings may beeffected either by controlling the zoom lens drive in the image pickupunit 3 or by getting the image pickup signal processing unit 15 suitablyprocessing signals.

(b) and (c) of FIG. 5 show cases where the image pickup capability blockacts as the image source that supplies the display image data to bedisplayed by the display unit 2. By contrast, FIG. 6 shows cases inwhich the storage unit 25 acts as the display image source that suppliesdisplay image data for display on the display unit 2.

More specifically, (a) of FIG. 6 shows a case where the storage unit 25retrieves moving or still image contents from its recording medium andhas the retrieved contents displayed by the display unit 2.

(b) of FIG. 6 shows a case in which the storage unit 25 has a video gameprogram activated from its recording medium and allows the image datacoming from the program to be displayed by the display unit 2.

(c) of FIG. 6 shows a case where the storage unit 25 has electronic bookcontents retrieved from its recording medium and enables the retrievedcontents to be displayed by the display unit 2.

In each of the cases of (a), (b) and (c) of FIG. 6 above, the user canenjoy image data reproduced from the recording medium through the use ofthe display apparatus 1.

FIGS. 7 and 8 show cases where the communication unit 26 acts as theimage source that supplies display image data for display on the displayunit 2.

FIG. 7 gives cases where the configuration of (b) of FIG. 2 or (d) ofFIG. 2 is in effect and where the display unit 2 displays the image datasent from the external image pickup apparatus 70 and received by thecommunication unit 26.

More specifically, (a) of FIG. 7 shows a case in which, with the user inthe situation of (a) of FIG. 5 watching a soccer game at a stadium, thedisplay unit 2 displays the image picked up by an image pickup apparatus70 positioned somewhere else in the same stadium and received by thecommunication unit 26. In this case, the user can watch the game withmore interest by getting the display unit 2 displaying images obtainedby the image pickup apparatus 70 located close to the team manager'sseat or images collected by a miniature image pickup apparatus 70 wornby the referee.

(b) of FIG. 7 shows a case in which the display unit 2 displays theimage picked up by an image pickup apparatus 70 set up at a resort or byan image pickup apparatus carried by an acquaintance going on a tour andreceived by the communication unit 26. In this case, the user can enjoyscenes of diverse regions or countries while at home.

(c) of FIG. 7 shows a case in which the display unit 2 displays aterrestrial view (bird's-eye view) picked up by an image pickupapparatus 70 mounted on aircraft or on a satellite and received by thecommunication unit 26. In this case, the user can enjoy scenes notencountered in daily life.

FIG. 8 shows cases where the configuration of (c) of FIG. 2 or (d) ofFIG. 2 is in effect and where the display unit 2 displays the image datasent from the external content source apparatus 71 and received by thecommunication unit 26.

More specifically, (a) of FIG. 8 shows a case in which the display unit2 displays image contents such as moving or still images received fromthe content source apparatus 71 such as an AV apparatus or a personalcomputer.

(b) of FIG. 8 shows a case in which the content source apparatus 71 suchas a personal computer sends to the display apparatus 1 image dataconstituting an onscreen display of a website accessed by a browserprogram of the apparatus 71 or image data making up the display of anapplication program being active on the apparatus 71. The displayapparatus 1 then has the image data received by the communication unit26 and displayed by the display unit 2.

(c) of FIG. 8 shows a case in which the content source apparatus 71 suchas a personal computer sends to the display apparatus 1 image dataconstituting the display of a list of photos or the like viewed on theapparatus 71. The display apparatus 1 then has the image data receivedby the communication unit 26 and displayed by the display unit 2.

In the preceding cases, the content source apparatus 71 is typically oneof AV apparatuses including video players or an information processingapparatus such as a personal computer. The display apparatus 1 receivesand displays the image data sent from any of these content sourceapparatuses 71. The user wearing the display apparatus 1 can then verifythe displayed images and perform various operations accordingly.

FIG. 6 shows cases where the storage unit 25 acts as the source ofincoming image data, whereas FIG. 8 indicates cases where thecommunication unit 26 serves as the image data source. Alternatively,the images shown in FIG. 8 may be considered to be reproduced imagescoming from the storage unit 25. As another alternative, the imagesindicated in FIG. 6 may be regarded as images that have been sent fromthe external apparatus and received by the communication unit 26.

FIGS. 9, 10 and 11 show cases in which images from the above-describeddiverse sources (image pickup capability block, reproduction capabilityblock, and reception capability block) are processed in terms of displayform or in image data format.

(a) of FIG. 9 shows a case where the display unit 2 is placed in thesee-through state.

(b) of FIG. 9 shows a case in which the display unit 2 displayssplit-screen images, with the system controller 11 giving asplit-display instruction to the display control unit (display imageprocessing unit 12, display drive unit 13). Illustratively, the onscreendisplay of the display unit 2 may be split into areas AR1 and AR2. Thearea AR1 may be in the see-through state or in the normal image displaystate, while the area AR2 may display images coming from the storageunit 25 or communication unit 26 acting as the source (the images may bereproduced or received video content images).

(c) of FIG. 9 shows another example of the split-screen display. In thiscase, the screen of the display unit 2 is illustratively split into fourareas AR1, AR2, AR3 and AR4, each area displaying one of the framesextracted at predetermined intervals from the image of interest.Illustratively, the display image processing unit 12 is arranged toextract image data on a frame by frame basis at intervals of 0.5seconds. The extracted image frames are displayed in cyclical fashionfrom AR1 to AR2 to AR3 to AR4 to AR1 to AR2, and so on. This is anexample of images picked up typically under strobe lighting forsplit-screen display on the display unit 2.

Obviously, it is also possible for each of a plurality of areas makingup the screen to display an image from each of different sources.

(a) of FIG. 10 shows a case where an image from the image pickupcapability block, reproduction capability block, or reception capabilityblock is displayed in normal size.

If the system controller 10 gives an image scale-up instruction to thedisplay image processing unit 12 through the display control unit 14,then the display unit 2 may display a scaled-up image such as one shownin (b) of FIG. 10.

(a) of FIG. 11 shows a case in which an image coming from the imagepickup capability block, reproduction capability block, or receptioncapability block is displayed as picked up.

This is an image at a low brightness level and it may not be easy forthe user to watch it comfortably.

In such a case, the system controller 10 may instruct the displaycontrol unit 14 (display image processing unit 12, display drive unit13) to increase brightness, adjust contrast, and/or enhance sharpness,thereby causing the display unit 2 to display a brighter, clearer imagesuch as one shown in (b) of FIG. 11.

The foregoing displays are merely examples and are not limitative of thepresent invention. Many other forms of displays can be implemented bythe embodiment of the invention. Specifically, such displays are broughtabout by selecting the source out of the image pickup capability block,reproduction capability block, and reception capability block asdesired; by suitably controlling the image pickup capability block orreproduction capability block in operation; or by properly controllingthe display image processing unit 12 or display drive unit 13 in termsof processing or in operation.

Illustratively, if the image pickup capability block is selected as thedisplay image source, then it is possible to implement the followingdisplays: telephoto view display, wide-angle view display, zoom-in andzoom-out displays shifting from telephoto to wide-angle settings or viceversa, scale-up display, scale-down display, display at variable framerates (e.g., images picked up at high or low frame rates), highintensity display, low intensity display, variable contrast display,variable sharpness display, display at enhanced imaging sensitivity,display at enhanced infrared imaging sensitivity, display at enhancedultraviolet imaging sensitivity, and display of images with theirspecific wavebands suppressed.

If the reproduction capability block or reception capability block isselected as the display image source, then it is possible to implementdisplays of images reproduced at varied speeds such as high-speedreproduction, slow reproduction, frame-by-frame advance reproduction; aswell as changing of pages being displayed or scrolling of images as iscarried out on a personal computer or the like.

Where the display image processing unit 12 is suitably controlled interms of processing, it is conceivable to implement displays withdiverse onscreen effects, such as images with pixelated mosaic effect,images in reverse video, soft-focus images, partially highlightedimages, and images in varied ambient colors.

4. Determination of User Status

As described above, the display apparatus 1 embodying the presentinvention has the visual sensor 19, acceleration sensor 20, gyro 21, andbiometric sensors 22 constituting the structure for acquiring userinformation.

The visual sensor 19 is used to detect information about the user'ssight. For example, the visual sensor 19 may be formed by an imagepickup arrangement located close to the display unit 2 to take picturesof the user's eyes. Images of the user's eyes are picked up by thatimage pickup arrangement and acquired by the system controller 10wherein the user status determination capability 10 a analyses theobtained images. The analysis provides information about the line ofsight, focal distance, pupil dilation, retinal pattern, and blinks ofthe user. The information thus acquired is used as the basis fordetermining the user's status or intentions.

Alternatively, the visual sensor 19 may be formed by a light-emittingarrangement and a light-receiving arrangement, the light-emittingarrangement being located close to the display unit 2 and emitting lightto the user's eyes, the light-receiving arrangement receiving reflectedlight from the user's eyes. Illustratively, the thickness of the user'scrystalline lens is detected from a received-light signal. The detectedlens thickness in turn serves as the basis for detecting the focaldistance of the user's eye.

By detecting the user's line of sight, the system controller 10 candetermine which part of the image displayed illustratively by thedisplay unit 2 is attracting the user's attention.

It is also possible for the system controller 10 to recognize the user'sline of sight as an input operation. For example, the user's line ofsight being moved rightward or leftward may be interpreted asinstructions input to the display apparatus 1.

By detecting the user's focal distance, it is possible to determinewhether the scene currently attracting the user's attention is nearby orfar away. This in turn permits execution of zoom control andscale-up/scale-down control. Illustratively, the user's act of lookingat a distant object may be interpreted as a cue to bring about atelephoto view display.

By detecting the degree of the user's pupil dilation, it is possible todetermine how the user is reacting to glare of the ambient brightness ifthe see-through state is in effect or to glare of the onscreen imagebrightness if the monitor display state is in use. According to theuser's reaction thus determined, the system controller 10 may controlbrightness and imaging sensitivity.

The user's retinal pattern may be detected illustration for use inauthenticating the user's identity. Since every person has his or herunique retinal pattern, the user wearing the display apparatus isidentified so that the apparatus may be controlled in a manner suitablefor the detected identity. Alternatively, the display apparatus mayenable its monitor to be operated only for a specifically authenticateduser.

By detecting the user's blinks, it is possible to determine the degreeof the user's eyestrain due to glare or other causes. It is alsopossible to interpret the user's blinks as input operations deliberatelyexecuted by the user. For example, three consecutive blinks made by theuser may be interpreted as a cue to bring about a particular operation.

The acceleration sensor 20 and gyro 21 output signals corresponding tothe user's movements. Illustratively, the acceleration sensor 20 issuitable for detecting linear movements while the gyro 21 is suited tosense rotary or vibratory motions.

When appropriately positioned on the user's body, the accelerationsensor 20 and gyro 21 can detect movements of the user's whole body orbody parts.

For example, the acceleration sensor 20 and gyro 21 may be mountedinside the eyeglass-like display apparatus 1 shown in FIG. 1 so as todetect movements of the user's head. In this case, the accelerationsensor 20 provides information about the acceleration of the user's heador whole body, and the gyro 21 yields information about the angularvelocity and vibration of the user's whole body.

The above sensor setup primarily detects the user's behavior in terms ofhead or neck movements. Illustratively, the sensors can detect whetherthe user is looking upward or downward. The user's head directeddownward may be interpreted to mean that the user is looking at a nearbyobject like a book as in reading; the user's head oriented upward may beinterpreted to signify that the user is looking at a faraway object.

Upon detecting the user shaking his or her head, the system controller10 can interpret the motion as the user's deliberate action. Forexample, if the user is found to shake his head twice leftward, theaction may be recognized as a particular input operation.

The acceleration sensor 20 and gyro 21 may be suitably arranged todetect whether the user is at rest (stationary), is walking, or isrunning. It is also possible for these sensors to determine whether theuser has sat down from his or her standing position or has now stood upfrom the sitting position.

As another alternative, the acceleration sensor 20 and gyro 21 may beattached to the user's arms or legs apart from the wearable unit mountedon the user's head. This sensor setup makes it possible to detectmovements of only the arms or the legs of the user.

The biometric sensors 22 are designed to detect information about theheart rate, pulse rate, perspiration, brain wave (e.g., alpha wave, betawave, theta wave, gamma wave), galvanic skin reflex, body temperature,blood pressure, and breathing (e.g., rate and depth of breathing, amountof ventilation) of the user. These items of information, when acquired,allow the system controller 10 to detect whether the user is in a tensestate, an excited state, a calm state, a comfortable state, or anuncomfortable state.

Whether or not the display apparatus 1 is currently worn by the user maybe determined based on biometric information, and this information maybe used for operational purposes. Illustratively, if the displayapparatus 1 is found unmounted by the user, then the system controller10 may place the apparatus 1 in standby mode in which checks are madeonly for biometric information. As soon as the display apparatus 1 isfound worn by the user, the system controller 10 may activate theapparatus 1. When the user is found to have unmounted the displayapparatus 1, the system controller 10 may place the apparatus 1 backinto standby mode.

The information detected by the biometric sensors 22 may also beutilized for authentication of the user's identity (so that theapparatus should be worn by the properly identified person).

The biometric sensors 22 may be located inside the head mount frame ofthe eyeglass-like display apparatus 1. Illustratively, the informationdescribed above may be derived from the temples and back of the user'shead. Alternatively, the biometric sensors 22 may be positionedelsewhere on the user's body, independent of the head mount frame of thedisplay apparatus 1.

5. Various Operations

As described above, the display apparatus 1 embodying the presentinvention has its system controller 10 control display-relatedoperations in accordance with the user information detected by thevisual sensor 19, acceleration sensor 20, gyro 21, and biometric sensors22. In this manner, the system controller 10 allows the display unit 2to display images that reflect the user's intention and status and arethus fit for his or her preferences.

Such operations carried out under control of the system controller 10will now be described below.

FIG. 12 is a flowchart of steps constituting a typical control processperformed by the operation control capability 10 b of the systemcontroller 10.

In step F101, the system controller 10 controls the display control unit14 to place the display unit 2 in the see-through state. Illustratively,when the display apparatus 1 is initially switched on, the systemcontroller 10 goes to step F101 and puts the display unit 2 in thesee-through state.

With the display unit 2 in the see-through state, the system controller10 goes to step F102 and checks to determine whether a display starttrigger is detected. The display start trigger signifies an event thatcues the system controller 10 in causing the display unit 2 to start itsdisplay operation in view of the user's intention or status determinedby the user status determination capability 10 a. The system controller10 recognizes a specific display start trigger in the form of the user'sexplicit operation, user's deliberate act (i.e., action interpreted asan operation), or the user's unconscious movement or status (includingthe user's perception). Some examples of the trigger are describedbelow.

If a display start trigger is detected, then the system controller 10goes to step F103 and controls the start of display. More specifically,the system controller 10 instructs the display control unit 14 tocontrol the display image processing unit 12 and display drive unit 13in order to have the display unit 2 display supplied data as a normalpickup image.

The display image processing unit 12 is supplied with image data throughthe image input/output control unit 27. If a plurality of image datasources exist as shown in FIG. 3 or 4, then the display image processingunit 12 may be fed at this point with the image data from the imagepickup capability block (image pickup unit 3, image pickup signalprocessing unit 15), reproduction capability block (storage unit 25), orreception capability block (communication unit 26), whichever isselected by default. Illustratively, if the image pickup capabilityblock is selected by default as the source, then the system controller10 reaching step F103 instructs the image pickup control unit 11 tostart picking up images, causes the image pickup unit 3 and image pickupsignal processing unit 15 to perform normal image pickup operation, andsends the image data constituting an image pickup signal to the displayimage processing unit 12 through the image input/output control unit 27.In this case, the display unit 2 is typically switched from thesee-through state of (a) of FIG. 5 to the monitor display stateeffecting a normal pickup image as shown in (b) of FIG. 5.

If the default source is the storage unit 25, then the system controller10 in step F103 may start up the storage unit 25 and get the displayunit 2 to display reproduced contents or a menu screen for selection ofcontents coming from the storage unit 25. If the default source is thecommunication unit 26, then the system controller 10 in step F103 maystart up the communication unit 26 and get the display unit 2 to displaya communication-ready screen or images of data received from an externalapparatus.

Obviously, if there is one source, the image data from that source needbe supplied to the display image processing unit 12.

As another alternative, when step F103 is reached in which to controlthe start of display, the display unit 2 may be arranged to display amenu screen or a source selection screen as its initial screen, with noimage data supplied from image sources.

Although the process of FIG. 12 contains no particular mention of audiooutput from the audio output unit 5, it is assumed that during thedisplay operation by the display unit 2, the system controller 10controls the audio input/output control unit 28 and audio signalprocessing unit 16 to output sounds based on the audio data from thesame source that supplies display images.

While the image coming from a given source is being displayed by thedisplay unit 2, the system controller 10 goes to step F104 and checks todetermine whether a display control trigger has occurred. If no displaycontrol trigger is detected, then the system controller 10 goes to stepF105 and checks to see if a source switch trigger has occurred. If nosource switch trigger is detected, then the system controller 10 goes tostep F106 and determines if a display end trigger has occurred.

The display control trigger signifies an event that cues the systemcontroller 10 in making changes to how images are to be displayed or howthe display image data is to be processed, in view of the user'sintention or status determined by the user status determinationcapability 10 a.

The source switch trigger means an event that cues the system controller10 in switching sources of display image data in view of the user'sintention or status determined by the user status determinationcapability 10 a, where the display apparatus 1 has a plurality ofsources as shown in FIG. 3 or 4.

The display end trigger refers to an event that cues the systemcontroller 10 in causing the display unit 2 to end its display operationand enter the see-through state in view of the user's intention orstatus determined by the user status determination capability 10 a.

The triggers above are detected by the system controller 10 asrepresentative of the user's deliberate operation (any action recognizedas an operation) or the user's unconscious operation or status (i.e.,user's physical or mental state). How such triggers are recognized andwhat will be controlled as a result of the detected trigger will bediscussed below in more detail.

If a display control trigger is detected, the system controller 10 goesfrom step F104 to step F107 and controls image display operation.Specifically, the system controller 10 instructs the display controlunit 14 to have the display unit 2 give a display reflecting the user'sintention or status at that point. Depending on the currently selectedsource, the system controller may control the image pickup capabilityblock, the storage unit 25 in operation, or the communication unit 26 inoperation.

Following the display operation control in step F107, the systemcontroller 10 continues to check for triggers in steps F104, F105 andF106.

If a source switch trigger is detected, the system controller 10 goesfrom step F105 to step F108 and controls source switching operation.Specifically, the system controller 10 instructs the image input/outputcontrol unit 27 and/or the audio input/output control unit 28 to controlthe newly switched source in operation and to have the display imageprocessing unit 12 and/or audio signal processing unit 16 supplied withthe image data and/or audio data coming from the new source.

The source switch control in step F108 illustratively switches thedisplay unit 2 from the state in which the images picked up by the imagepickup unit 3 have been displayed, to the state in which the imagesreproduced from the storage unit 25 are displayed.

Following the source switch control in step F108, the system controller10 continues to check for triggers in steps F104, F105 and F106.

If a display end trigger is detected, then the system controller 10 goesfrom step F106 back to step F101 and instructs the display control unit14 to put the display unit 2 in the see-through state. The currentlyselected image data source is instructed to end its image feedingoperation.

While the display apparatus 1 remains active and is worn by the user,the operation control capability 10 b of the system controller 10carries out the process outlined in FIG. 12.

During the process, four kinds of controls are effected: display startcontrol following detection of the display start trigger, displayoperation control following detection of the display control trigger,source switch control following detection of the source switch trigger,and control to put the display unit 2 in the see-through state followingdetection of the display end trigger. Further details of the checks fortriggers and specific controls to be executed following triggerdetection will be discussed below in reference to FIG. 13 and subsequentfigures.

FIGS. 13 through 19 outline typical processes performed by the userstatus determination capability 10 a of the system controller 10. Theseprocesses are assumed to be executed in parallel with the process ofFIG. 12 performed by the operation control capability 10 b. What theparallel execution illustratively signifies is that the process of FIG.12 being carried out by the system controller 10 is periodicallyinterrupted for detection of triggers by the processes of FIGS. 13through 19. The programs of these processes may be either embedded inthe program of the process in FIG. 12 or furnished separately from thelatter program and invoked periodically. That is, the programs may beprovided in any suitable form.

FIG. 13 shows cases in which a display start trigger is detected as acue to switch from the see-through state to the display state.

(a) and (b) of FIG. 13 indicate cases where the user's behavior ischecked for the trigger by which to initiate monitor display.

In step F200 of (a) of FIG. 13, the system controller 10 checks forinformation detected by the acceleration sensor 20 and/or gyro 21(acceleration signal, angular velocity signal).

It is assumed that the user's specific movement, such as moving down hishead twice, shaking his head right and left in one stroke, or rotatinghis head in a circular movement, is predetermined as representative ofthe user's decision to start the display operation. If the detectioninformation from the acceleration sensor 20 and/or the gyro 21 is foundto indicate the user's specific moment designating the start of display,then the system controller 10 goes from step F201 to step F202 andrecognizes a display start trigger.

With the display start trigger detected in step F202, control of theprocess in FIG. 12 is passed from step F102 to step F103. In step F103,the system controller 10 controls the display unit 2 to start its imagedisplay.

The user's specific behavior to be detected by the acceleration sensor20 or gyro 21 as representative of the user's decision to start monitordisplay is not limited to the example cited above. Alternatively, theuser may jump, wave his hands, swing his arms, swing his legs, orperform any other bodily motion to gain the same result.

(b) of FIG. 13 shows a case where a display start trigger is determinedon the basis of information from the visual sensor 19.

In step F210, the system controller 10 analyzes the information comingfrom the visual sensor 19. Illustratively, if the visual sensor 19includes an image pickup device to pick up images of the user's retinalpattern, then the system controller 10 analyzes the image thus acquiredby the visual sensor 19.

If the user's three consecutive blinks are predetermined asrepresentative of the user's decision to start the display operation,then the system controller 10 checks for that particular behaviorthrough image analysis. Upon detecting three consecutive blinks of theuser, the system controller 10 goes from step F211 to F212 andrecognizes a display start trigger.

With the display start trigger detected in step F212, control of theprocess in FIG. 12 is passed from step F102 to step F103.

In step F103, the system controller 10 controls the display unit 2 tostart its image display.

The user's specific behavior to be detected by the visual sensor 19 asrepresentative of the user's decision to start monitor display is notlimited to the example cited above. Alternatively, the user may roll theeyes in a circular motion, move the eyes right and left or up and downin two strokes, or do any other suitable eye movement to gain the sameresult.

(c) of FIG. 13 shows a case in which the user's act of wearing thedisplay apparatus 1 itself is interpreted as a display start trigger.

In step F230, the system controller 10 checks for such information asbrain wave, heart rate, and galvanic skin reflex from the biometricsensors 22.

In step F231, the system controller 10 checks the information from thebiometric sensors 22 to determine whether the user is wearing thedisplay apparatus 1. The fact that biometric information starts gettingacquired by the biometric sensors 22 is interpreted as the user startingto wear the display apparatus 1.

Upon detecting the user wearing the display apparatus 1, the systemcontroller 10 goes from step F231 to step F232 and recognizes a displaystart trigger. With the display start trigger detected in step F232, thesystem controller 10 reaches step F103 in FIG. 12 and carries out thedisplay start control.

Whether or not the user is wearing the display apparatus 1 is determinedas described above on the basis of the vital reaction information fromthe biometric sensors 22.

As soon any of such findings as the pulse rate, brain wave, and galvanicskin reflex starts getting detected, the display start trigger isrecognized. The user's act of wearing the display apparatus 1 thusprompts the system controller 10 to start the display operation.

It is also possible to start display control when the display apparatus1 is worn not by any user but by a particular user. The user's retinalpattern detected by the visual sensor 19 or the detection signal fromthe biometric sensors 22 may be used to identify individual users. Wherethe retinal patterns or specific biometric information about expectedusers is registered in advance, the system controller 10 may determinewhich user is currently wearing the display apparatus 1 in accordancewith the incoming information.

Specifically, when a certain user wears the display apparatus 1, thesystem controller 10 authenticates the user's identity. If a particularuser's identity is recognized, then the system controller 10 detects thedisplay start trigger and performs display start control accordingly.

It is also possible to detect the display start trigger by interpretingthe user's diverse unconscious behavior or physical status.

For example, a sudden shift in the user's line of sight in theinformation coming from the visual sensor 19 may be interpreted as adisplay start trigger. The display start trigger may be recognizedalternatively by use of other information from the visual sensor 19 oron the basis of input sounds from the audio input unit 6.

When an image is displayed by the display unit 2 in response to thedisplay start trigger as described above, a see-through state area AR1may be left intact on the screen of the display unit 2 while anotherarea AR2 is arranged to display the image being picked up, as shown in(b) of FIG. 9.

Although not shown in FIG. 12, arrangements may be made to recognize apower-on trigger from the user starting to wear the display apparatus 1as shown in (c) of FIG. 13. In this case, the system controller 10 turnson the display apparatus 1 upon detection of the apparatus 1 gettingworn by the user.

Conversely, the system controller 10 may switch off the displayapparatus 1 when detecting the apparatus 1 getting unmounted by theuser.

Described below in reference to FIGS. 14 through 17 are typicalprocesses carried out as a result of the recognition of a displaycontrol trigger in step F104 of FIG. 12.

FIG. 14 is a flowchart of steps constituting the typical process formoving the displayed page or its highlighted portion forward andbackward in response to the user's line of sight movements.

In step F300 of FIG. 14, the system controller 10 analyzes theinformation coming from the visual sensor 19. If the visual sensor 19includes an image pickup unit to take pictures of the user's eyes, thenthe system controller 10 analyzes the eye image thus acquired.

If the user's line of sight is found moving leftward, then the systemcontroller 10 goes from step F301 to step F303 and recognizes animage-back trigger.

If the user's line of sight is found moving rightward, then the systemcontroller 10 goes from step F302 to step F304 and recognizes animage-forward trigger.

In any of the cases above, control of the process in FIG. 12 is passedfrom step F104 to step F107 and the system controller 10 performsdisplay image forward/backward control. If the storage unit 25 is set tobe the source, with the reproduced image displayed as shown in (c) ofFIG. 6 or (c) of FIG. 8, then the system controller 10 controls thedisplay image processing unit 12 and storage unit 25 in such a manner asto move forward or back the current page or its highlighted portion.

Although the example above was shown to utilize detection of the user'sline of sight moving right or left, this is not limitative of thepresent invention. Alternatively, the user's line of sight moving up ordown may be detected and interpreted as a screen scroll trigger, wherebythe display screen may be scrolled as desired by the user.

FIG. 15 is a flowchart of steps constituting the typical process forcontrolling display status based on the user's pleasant and unpleasantsensations.

In step F400 of FIG. 15, the system controller 10 analyzes informationcoming from the visual sensor 19 to detect the user's pupil dilation andblinks (number of blinks per unit time).

In step F401, the system controller 10 checks biometric information suchas the brain wave, heart rate, perspiration, and blood pressure comingfrom the biometric sensors 22.

Given such information from the visual sensor 19 and biometric sensors22, the system controller 10 checks to determine whether the user is ina comfortable state with regard to the image being displayed by thedisplay unit 2.

If the user's state of visual perception is found to be uncomfortable,then the system controller 10 goes from step F402 to step F403 andrecognizes an image adjustment trigger. The system controller 10 thengoes to step F404 and calculates modified settings deemed to bring aboutcomfort to the user in his or her situation, such as adjusted settingsof display brightness, contrast, and sharpness.

Following steps F403 and F404, control of the process in FIG. 12 by thesystem controller 10 is passed from F104 to step F107. In this case, thesystem controller 10 instructs the display image processing unit 12 tocarry out such processes as brightness level adjustment, contrastadjustment, and sharpness control. The processing adjusts the quality ofthe image displayed by the display unit 2 so that the image will appearmore comfortable to the user than before.

For example, the user's state of visual perception may deteriorate byeyestrain or as a result of aberrant ambient brightness against whichthe image picked up by the image pickup capability block is beingdisplayed. In such a case, the above processing corrects the situationand allows the user to restore a pleasant state of visual perception.Illustratively, a dimmed unclear image such as one shown in (a) of FIG.11 may be replaced by a clearer image indicated in (b) of FIG. 11. Incase of eyestrain on the user's part, the currently displayed image maybe turned from its hard contrast state into an image of softer contrast.

FIG. 16 is a flowchart of steps constituting the typical process forinterpreting movements of the user's head as meaningful operations.

In step F500, the system controller 10 checks for information detectedby the acceleration sensor 20 and/or gyro (acceleration signal, angularvelocity signal). In step F501, the system controller 10 determines themovement of the user's head. For example, checks are made to see if theuser's head has moved backward twice, forward twice, or leftward twice.

If the user's head is found to have moved backward twice, then thesystem controller 10 goes from step F502 to step F505 and recognizes animage scale-up trigger.

In this case, control of the process in FIG. 12 by the system controller10 is passed from step F104 to step F107, and the system controller 10instructs the display image processing unit 12 to perform an imagescale-up process. This in turn causes the display unit 2 to display ascaled-up image such as one in (b) of FIG. 10.

If the user's head is found to have moved forward twice, then the systemcontroller 10 goes from step F503 to step F506 and recognizes an imagescale-down trigger. In this case, control of the process in FIG. 12 bythe system controller 10 is also passed from step F104 to step F107, andthe system controller 10 instructs the display image processing unit 12to perform an image scale-down process. This in turn causes the displayunit 2 to display a scaled-down image.

If the user's head is found to have moved leftward twice, then thesystem controller 10 goes from step F504 to step F507 and recognizes animage scale reset (default) trigger. In this case, too, control of theprocess in FIG. 12 by the system controller 10 is passed from step F104to step F107, and the system controller 10 instructs the display imageprocessing unit 12 to perform a scale reset process. This in turn causesthe display unit 2 to display an image of the default scale.

FIG. 17 is a flowchart of steps constituting the typical process forrecognizing a trigger for making input operations on the display screenbased on the information from the visual sensor 19.

In step F600 of FIG. 17, the system controller 10 analyzes theinformation coming from the visual sensor 19. If the visual sensor 19includes an image pickup unit to take pictures of the user's eyes, thenthe system controller 10 analyzes the eye image thus acquired and checksfor movements of the user's line of sight or for the user's blinks whilelooking at the display unit 2.

Upon detecting a movement of the user's line of light, the systemcontroller 10 goes from step F601 to step F603 and recognizes a triggerfor moving the cursor (i.e., onscreen pointer) on the display screen. Instep F604, the system controller 10 calculates the direction and amountof the cursor movement over the screen based on the direction and mountof the detected line-of-sight movement.

In this case, the system controller 10 reaches step F107 in FIG. 12 andinstructs the display image processing unit 12 to move the cursor ascalculated. Illustratively, if the image such as one in (b) and (c) ofFIG. 8 is being displayed, the cursor is moved on that display screen.

Upon detecting the user's two consecutive blinks, the system controller10 goes from step F602 to step F605 and recognizes a trigger for a clickoperation. In this case, the system controller 10 reaches step F107 inFIG. 12 and performs a control process corresponding to the click.Illustratively, if the click is found to be associated with a reproducedimage coming from the storage unit 25, the system controller 10 controlsthe storage unit 25 in its reproducing operation in a manner reflectingthe click operation; if the click is found to be made on the image beingreceived by the communication unit 26, the system controller 10 sends tothe connected external apparatus the cursor position and click-invokedoperation information in effect at the moment.

The processing above allows the user's eye movements to be used astriggers commensurate with the operations of a mouse attached to atypical computer system. When a video game image such as one in (b) ofFIG. 6 is being displayed, the user's line of sight and blinks may bedetected and utilized as the user's operations reacting to the game inprogress.

The processes in FIGS. 14, 15, 16 and 17 were shown to control howimages are to be displayed onscreen, how the image data for display isto be processed, or how operations are to be carried out on the displayscreen in response to the user's deliberate or unconscious operations orphysical status. However, the examples cited above are only forillustration purposes; there may be many other examples in which displaycontrol triggers are detected and controls are carried out accordingly.

A display control trigger can be recognized whenever predeterminedconditions are found to be met following detection of the user'sbehavior and physical status from diverse information coming from thevisual sensor 19, acceleration sensor 20, gyro 21, and biometric sensors22.

When a display control trigger is recognized under diverse conditions,the display image processing unit 12 may be operated to executescale-up/scale-down control; brightness, contrast and sharpnessadjustment; control on image effects including pixelated mosaic effect,images in reverse video, and soft-focus images; and control onsplit-screen display and strobe-induced image display, in accordancewith the user's behavior and physical status in effect.

If the image pickup capability block is selected as the source, withpicked-up images displayed by the display unit 2, then the image pickupunit 3 and image pickup signal processing unit 15 may be operated toexecute zoom control between telephoto and wide-angle shot settings,imaging sensitivity control, switchover between imaging frame rates, andchanges to infrared or ultraviolet imaging sensitivity settings.

If the reproduced image from the storage unit 25 or the received imagefrom the communication unit 26 is displayed by the display unit 2, thendiverse display control triggers may be recognized and the storage unit25 or communication unit 26 may be operated accordingly. That is, theoperations may deal with variable speed reproduction controls such asfast-forward/fast-rewind, instant access to desired positions,frame-by-frame feed, slow reproduction, and pause; page feed, pagescroll, shift of a highlighted portion within a list display, cursormovement, decision-finalizing operation, and video game actions.

In other words, the conditions under which to detect display controltriggers may be associated with the corresponding control operations ina large number of combinations.

When onscreen displays on the display unit 2 are to be changed inresponse to the display control trigger as described above, the widerarea AR1 may be left unchanged in its see-through state or normaldisplay sate while the smaller area AR2 is arranged to display images ina different form, as shown in (b) of FIG. 9. Obviously, it is alsopossible to display an image reflecting the display control trigger inthe wider area AR1. As another alternative, the screen may be dividedinto equal parts in which the normal pickup image and the imagecorresponding to the display control trigger are displayed as needed.

The foregoing description given in reference to FIGS. 14 through 18 wasabout how different examples of the display control trigger arerecognized and how they are handled through appropriate controls.Similarly, various examples of the source switch trigger to be detectedin step F105 of FIG. 12 may be conceived.

That is, the user's deliberate actions as well as his unconsciousbehavior and physical status may be detected from the informationsupplied by the visual sensor 19, acceleration sensor 20, gyro 21, andbiometric sensors 22. When a particular condition is found to be met,the presence of a source switch trigger may be detected.

With the source switch trigger thus recognized, the system controller 10may go to step F108 in FIG. 12 and switch the sources that supply imagedata to the display image processing unit 12.

Described below in reference to FIGS. 18 and 19 are examples of thedisplay end trigger to be detected in step F106 of FIG. 12, the triggergiving a cue to switch from the image display state to the see-throughstate.

(a) of FIG. 18 is a flowchart of steps constituting a typical processfor ending the display operation based on the user's deliberatebehavior.

In step F800 of (a) of FIG. 18, the system controller 10 checks theinformation detected by the acceleration sensor 20 and gyro 21 in orderto determine the user's head movement and whole body movement.

Upon detecting the user's particular behavior designating the desire toend monitor display, the system controller 10 goes from step F801 tostep F802 and recognizes a display end trigger.

After step F802, control of the process in FIG. 12 by the systemcontroller 10 is passed from step F106 to step F101. In this case, thesystem controller 10 instructs the display control unit 14 to effectswitchover to the see-through state. This puts the display unit 2 backinto the see-through state as shown in (a) of FIG. 5.

(b) of FIG. 18 is a flowchart of steps constituting another typicalprocess for ending the display operation based on the user's deliberatebehavior.

In step F810 of (b) of FIG. 18, the system controller 10 analyzesinformation coming from the visual sensor 19. If the user's threeconsecutive blinks were preset as an operation to be made by the user todesignate an end of display, then the system controller 10 checks forthis behavior through image analysis.

Upon detecting three consecutive blinks on the user's part, the systemcontroller goes from step F811 to step F812 and recognizes a display endtrigger.

After step F812, control of the process in FIG. 12 by the systemcontroller 10 is passed from step F106 to step F101, and the systemcontroller 10 instructs the display control unit 14 to effect switchoverto the see-through state. The display unit 2 is then placed back intothe see-through state as shown in (a) of FIG. 5.

Where the user's desire to bring about the see-through state isexpressed by behavior as outlined in (a) and (b) of FIG. 18, the controloperation reflecting the user's intention is carried out. Obviously, itis possible to conceive many other kinds of behavior on the user's partexpressing the user's desire to put the see-through state into effect.

(a) of FIG. 19 is a flowchart of steps constituting a typical processfor automatically switching back to the see-through state in response tothe user's movement (i.e., an action performed unconsciously as anactuating operation).

In step F900 of (a) of FIG. 19, the system controller 10 checks theinformation detected by the acceleration sensor 20 and gyro 21 in orderto determine the user's whole body movement. In particular, the systemcontroller 10 checks to determine whether the user is at rest, walking,or running.

Upon detecting the user starting to walk or run, the system control goesfrom step F901 to step F902 and recognizes that a display end triggerhas occurred.

After step F902, control of the process in FIG. 12 by the systemcontroller 10 is passed from step F106 to step F101, and the systemcontroller 10 instructs the display control unit 14 to effect switchoverto the see-through state. In turn, the display unit 2 is placed backinto the see-through state as shown in (a) of FIG. 5.

Interpreting the user's walking or running state as the cue to bringback the see-through state is desirable in terms of ensuring the user'ssafety.

However, the switch back to the see-through state on cue is notlimitative of the invention. Alternatively, upon detection of the user'swalking or running state, monitor display may be switched to the normalpickup image equivalent to the see-through state as shown in (b) of FIG.5.

(b) of FIG. 19 is a flowchart of steps constituting a typical processfor automatically bringing back the see-through state in response to theuser's physical status.

In step F910 of (b) of FIG. 19, the system controller 10 checks suchinformation from the biometric sensors 22 as the user's brain wave,heart rate, perspiration, and blood pressure.

In step F911, the system controller 10 analyses information from thevisual sensor 19 to check the user's pupil dilation and blinks (numberof blinks per unit time).

On the basis of the information coming from the biometric sensors 22 andvisual sensor 19, the system controller 10 checks to determine whetherthe user's eyes are unduly strained.

Upon detecting the user's eyestrain, the system controller 10 goes fromstep F912 to step F913 and recognizes that a display end trigger hasoccurred.

After step F913, control of the process in FIG. 12 by the systemcontroller 10 is passed from step F106 to step F101, and the systemcontroller 10 instructs the display control unit 14 to effect switchoverto the see-through state. Illustratively, display of the pickup image onthe monitor screen at enhanced infrared imaging sensitivity isterminated, and the display unit 2 is placed back into the see-throughstate.

When the display operation of the display unit 2 is brought to an end inresponse to the user's physical status such as eyestrain, the displayapparatus 1 can be used in a manner minimizing the burdens on the user'sphysical condition.

6. Effects, Variations and Extensions of the Embodiments

The preferred embodiment of the present invention has been describedabove. The description has focused, among others, on the display unit 2which is part of an eyeglass-like or head mount-type wearable unit ofthe display apparatus 1 and which displays images immediately before theuser's eyes. With this setup, information about the user's movements andphysical status is used as the basis for determining the user'sintentions and biometric status. The result of what has been determinedis in turn used to control display-related operations, whereby asuitable display is implemented corresponding to the user's intentionsand status with little operational effort on the user's part. Theinventive display apparatus 1 thus provides excellent usability andoffers the user varieties of visual experiences.

With the display unit 2 placed in the transparent or translucentsee-through state through control of its transmittance, the displayapparatus 1 when worn by the user does not bother him in his dailypursuits. The benefits of the inventive display apparatus 1 can beenjoyed by the user while leading ordinary life.

In the foregoing description, emphasis was placed on how to controldisplay operations. Alternatively, the user's behavior and biometricstatus may be used as the basis for controlling the switchover betweenpower-on, power-off, and standby state of the apparatus and foradjusting the volume and quality of sounds being output by the audiooutput unit 5. Illustratively, sound level control may be effected bychecking the information from the biometric sensors 22 to see howcomfortable the user is at present.

The display apparatus 1 may incorporate a character recognition blockthat recognizes characters in images, and a speech synthesis block thatperforms speech synthesizing processes. With such modifications inplace, the character recognition block recognizes any characters thatmay be found included in the images that have been picked up,reproduced, or received. Given the recognized characters, the speechsynthesis block generates read-aloud voice signals causing the audiooutput unit 5 to read the characters aloud.

The appearance and structures of the display apparatus 1 are not limitedto those shown in FIGS. 1, 3 and 4. Many other variations andmodifications are conceivable.

Although the display apparatus 1 was described as mountable on theeyeglass-like or head mount-type wearable unit, this is not limitativeof the invention. The display apparatus 1 according to the inventionneed only be structured to give display immediately before the user'seyes. Illustratively, the inventive apparatus may be of a headphonetype, a neck band type, a ear-hook type, or any other wearable type. Asother alternatives, the display apparatus may be worn by the user in amanner attached to the user's eyeglasses, visor, headphones, or otherdaily-worn accessories with a clip or like fittings.

1. (canceled)
 2. A display apparatus comprising: a display that displaysan image and is positioned in front of an eye of a user; a detectioncircuitry configured to detect a movement of the user; and a controllercircuitry configured to determine a status of the user based on thedetected movement and change the displayed image in response to thedetermined status of the user, the status of the user including one ofstationary and moving.
 3. The display apparatus according to claim 2,wherein the detection circuitry is further configured to detectmovements of the user that correspond to each of when the user isstationary, walking, and running.
 4. The display apparatus according toclaim 2, wherein the detection circuitry is further configured to detectthat the user's head has moved backward.
 5. The display apparatusaccording to claim 2, wherein: the controller circuitry is furtherconfigured to detect a display start trigger condition based on at leastone of information from a motion sensor, information from a visualsensor, and information from a biometric sensor; and the display iscontrolled by the controller circuitry to start displaying in accordancewith the detected display start trigger condition.
 6. The displayapparatus according to claim 2, further comprising: a biometric sensorconfigured to detect, as biometric information of the user, a heartrate, pulse rate, perspiration, brain wave, galvanic skin reflex, bloodpressure, body temperature, or breathing rate of the user.
 7. Thedisplay apparatus according to claim 6, wherein the controller circuitryauthenticates an identity of the user based on the biometric informationof the user detected by the biometric sensor.
 8. The display apparatusaccording to claim 2, wherein the detection circuitry includes anacceleration sensor and a gyro.
 9. The display apparatus according toclaim 3, wherein the controller circuitry is further configured to stopthe display if the detection circuitry detects the movement of the userthat corresponds to when the user is walking or when the user isrunning.
 10. The display apparatus according to claim 2, wherein thecontroller circuitry is further configured to start displaying a menuscreen or a source selection screen on the display as an initial screen.11. The display apparatus according to claim 2, wherein the controllercircuitry is further configured to control the display based on adetected orientation of the user's head.
 12. A method of displayingimages comprising: displaying an image on a display positioned in frontof an eye of a user; detecting a movement of the user; determining astatus of the user based on the detected movement; and changing thedisplay in response to the determined status of the user, the status ofthe user including one of stationary and moving.
 13. The methodaccording to claim 12, wherein: the detecting includes detectingmovements of the user that correspond to each of when the user isstationary, walking, and running.
 14. The method according to claim 12,further comprising: detecting that the user's head has moved backward.15. The method according to claim 12, further comprising: detecting adisplay start trigger condition based on at least one of informationfrom a motion sensor, information from a visual sensor, and informationfrom a biometric sensor; and controlling the display to start displayingin accordance with the detected display start trigger condition.
 16. Themethod according to claim 12, further comprising: detecting, asbiometric information of the user, a heart rate, pulse rate,perspiration, brain wave, galvanic skin reflex, blood pressure, bodytemperature, or breathing rate of the user.
 17. The method according toclaim 16, further comprising: authenticating an identity of the userbased on the biometric information of the user detected by thedetecting.
 18. The method according to claim 12, wherein: the detectingthe movement of the user detects the movement of the user using anacceleration sensor and a gyro.
 19. The method according to claim 13,further comprising: controlling the display to stop displaying if thedetecting detects the movement of the user that corresponds to when theuser is walking or when the user is running.
 20. The method according toclaim 12, further comprising: controlling the display to startdisplaying a menu screen or a source selection screen as an initialscreen.
 21. The method according to claim 12, further comprising:controlling the display based on a detected orientation of the user'shead.